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Abstract 

Traditional INSERTION Sort runs in 0{n 2 ) time because each insertion takes 0{n) time. When 
people run INSERTION Sort in the physical world, they leave gaps between items to accelerate inser- 
tions. Gaps help in computers as well. This paper shows that GAPPED INSERTION Sort has insertion 
times of 0(log n) with high probability, yielding a total running time of 0(n log n) with high probability. 
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1 Introduction 

Success has its problems. While many technology companies are hemorrhaging money and employees, 
Google is flush with money and hiring vigorously. Google employees are cheerful and optimistic, with the 
exception of G — . 

G — maintains the mailboxes at Google. The mailbox technology consist of trays arranged in linear 
order and bolted to the wall. The names on the mailboxes are alphabetized. G — is grumpy after each new 
hire because, to make room for the nth new employee, the names on 0(n) mailboxes need to be shifted by 
one. 

University graduate programs in the US have also been growing vigorously, accepting as students those 
talented employees downsized from high-tech companies. 

At Stony Brook S — implements the mailbox protocol. Each time a new student arrives, S — makes room 
for the new student's mailbox using the same technique as G — . However, S — only needs to shift names 
by one until a gap is reached, where the empty mailbox belonged to a student who graduated previously. 
Because the names have more or less random rank, S — does not need to shift many names before reaching 
a gap. 

Both S — and G — are implementing INSERTION Sort. However, while S — is blissfully unaware that 
Insertion Sort is an 0(n 2 ) algorithm, G — continually hopes that each new hire will be named Zhang, 
Zizmor, or Zyxt. 
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R — , the librarian at Rutgers, is astonished by all the fuss over insertions. R — inserts new books into the 
stacks 1 every day. R — plans for the future by leaving gaps on every shelf. Periodically, R — adds stacks to 
accommodate the growing collection. Although spreading the books onto the new stacks is laborious, these 
rearrangements happens so infrequently that R — has plenty of time to send overdue notices to hard-working 
students and professors. 

This paper shows that Gapped Insertion Sort, or Library Sort, has insertion times of O(logre) 
with high probability, yielding a total running time of 0(n log n) with high probability. 

Standard Insertion Sort 

In standard INSERTION SORT we maintain an array of elements in sorted order. When we insert a new 
element, we find its target location and slide each element after this location ahead by one array position to 
make room for the new insertion. The ith insertion takes time 0(i), for a total of 0(n 2 ). Finding the target 
position of the ith element takes time O(logz) using binary search, though this cost is dominated by the 
insertion cost. 

Library Sort 

We achieve 0(log ?7,)-time insertions with high probability by keeping gaps evenly distributed between the 
inserted elements and randomly permuting the input. Then we only need to move a small number of elements 
ahead by one position until we reach a gap. The more gaps we leave, the fewer elements we move on 
insertions. However, we can tolerate a small-constant-space overhead. 

The remainder of this paper is organized as follows. We present the details of the algorithm in Section 2 
and show in Section 3 that the algorithm runs in 0(n log n) time with high probability. In Section 4 we 
conclude with a few comments and some related work. 

2 Library Sort: Algorithm and Terminology 

Let A be an re-element array to be sorted. These elements are inserted one at a time in random order into a 
sorting array S of size (1 + e)n. The insertions proceed in log n rounds as follows. Each round doubles the 
number of elements inserted into S and doubles the prefix of S where elements reside. Specifically, round 
i ends when element 2 % is inserted and the elements are rebalanced. Before the rebalance, the 2* elements 
are in the first (1 + e)2 l positions. A rebalance moves them into the first (2 + 2e)2 l positions, spreading the 
elements as evenly as possible. We call 2 + 2e the spreading factor. 

During the ith round, the 2*" 1 elements in S at the beginning of the round are called support elements, 
and their initial positions are called support positions. The 2 1 " 1 elements inserted before the end-of -round 
rebalance are called intercalated elements. 

The insertion of 2 4_1 intercalated elements within round i is performed the brute force way: search for 
the target position of the element to be inserted by binary search (amongst the 2 l ~ 1 support positions in S), 
and move elements of higher rank to make room for the new element. Not all elements of higher rank need 
to be moved, only those in adjacent array positions until the nearest gap is found. 

3 Analysis 

For the sake of clarity, we divide the time complexity into four parts: the rebalance cost, the search cost, 
the insertion cost for the first y/n elements, and the insertion cost for the remaining elements. Let m be the 

'Throughout, we mean library stacks, an ordered set of stacks, rather than lifo stacks. 
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number of elements inserted at any time. 

3.1 Insertion cost for m = 0(^/n), rebalance cost, and search cost 
Lemma 1 The insertion time for the first 0(y/n) insertions is 0(n). 

Proof. By the quadratic running time of INSERTION SORT. □ 
Lemma 2 For a given input of size n, the cost of all rebalances is 0(n). 

Proof. Since the number of elements doubles in each round and the spreading factor is constant, the cost 
of spreading m elements on each rebalance is amortized over the previous m/2 insertions, for an amortized 
rebalance cost of 0(1) per insertion. □ 

Lemma 3 The cost of finding the location to insert a new element in the sorting array is 0(log m). 

Proof. Binary search among the 0(m) support positions takes time O(logm). A final search between 
two support positions takes time O(l), since the spreading factor is constant. □ 

3.2 Insertion cost for m = 17 (v^) 

We now bound the number of elements moved per insertion when m = fl(y/n) elements have already 
been inserted. We show that with high probability, for sufficiently large c, all sets of c log m contiguous 
support elements have fewer than (1 + e) clogm intercalated elements inserted among them by the end of 
the round. The clogm support elements are spread among (2 + 2e) clogm sorting array positions at the 
beginning of a round. Therefore, after (1 + s)c log m intercalated elements are added, there will still be gaps 
— indeed, there will be eclog m empty array positions. Thus, each insertion takes time 0(log m) because 
shifts propagate until the next gap, which appears within O(logm) positions. This observation establishes 
our result. 

The direct approach 

Let D be a set of c log m contiguous support elements. We would like to compute the number of intercalated 
elements that land among the elements of D. Notice that if there are k elements in the sorting array, then the 
k+ 1st intercalated element is equally likely to land between any of those k elements. Thus, if an intercalated 
element is inserted within D, the probability of further insertions within D increases, and conversely, if an 
intercalated element is inserted outside of D, the probability of further insertions within D decreases. 

We formalize the problem as follows. Consider two urns, urn A starting with clogm balls and urn B 
starting with m — c log m balls. Throw m additional balls, one after another, into one of the two urns with 
probability proportional to the number of balls in each urn. Let random variable X{ = 1 if ball i lands in 
urn A and let Xi = if ball i lands in urn B. We now need to bound the tails of^Xi. 

Because these Xj are positively correlated, bounding the tail of their sum is awkward. We analyze a 
simpler game below. 
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The arrival permutation 

We first set up the problem. Consider 2m elements to sort. Each of the (2m) ! orders of insertion is equally 
likely. We refer to each insertion order as an arrival permutation. The first half of the arrival permutation 
consists of support elements, and the second half consists of intercalated elements. Thus, the probability of 
being a support (resp. intercalated) element equals the probability of being in the first (resp. second) half of 
the arrival permutation. 

Our goal is to show that for sufficiently large c, with high probability in every set of (2 + e)c\ogm 
contiguous elements, there are at least c log m support elements and at least c log m intercalated elements at 
the end of a round. Thus, with high probability there are also at most (1 + e)clog m of each type in every 
set of (2 + e)clog m contiguous elements. Because the at least clogm support elements are spread out in 
a subarray of size (2 + 2e)clogm, there is room to add the at most (1 + e)clogm intercalated elements 
while still leaving gaps. Therefore, with high probability no insertion will move more than (2 + e)clogm 
elements. 

Theorem 4 In any set C of {2 + e)clog m contiguous elements, there are at least clog m support elements 
and at least c log m intercalated elements with high probability. 

Proof. Consider choosing an arrival permutation V of length 2m uniformly at random by placing the 
elements one-by-one into V, selecting an empty slot uniformly at each step. We place the elements of set 
C into V before placing the elements of C in V. We give an upper bound on the number of elements in C 
that are support elements, that is, the number of elements that fall in the first half of V. The argument for 
intercalated elements is symmetric. 

Let Si be the number of elements already inserted into the first half of V just before the ith insertion. 
The probability p, L that the ith insertion is in the first half of V is then (m — Sj)/ (2m — i + 1). 

Let random variable Xi = 1 if element i is a support element, and let Xi = otherwise. Random 
variables X\ , . . . , X2 m now depend on the remaining ] blank spaces in the permutation and are negatively 
correlated. Furthermore, |C| = (2 + e)c\ogm is small compared to m, and so -EpQ] = pi is very close to 
1/2 for the first \ C\ element. Given this bound, we can prove our theorem with a straightforward application 
of Chernoff bounds. 

Here we prove the theorem using elementary methods, as follows. The probability that a given element 
in C is a support element is at most 



m 



m 



2m-\C\ + l 2m - (2 + e)clogm + 1 



Let p be the probability that there are at most c log m support elements in the set C. Then, 




c log m 



P 




3=0 



Bounding the summation by the largest term, we obtain 




Using Stirling's approximation, for some constant d we obtain 
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P < c ( m ) ^/cIq 



r<l / rr. , \ \ clogm 

1^1 /( 2 + e )(2+e)\ 



; m 



,2m-|C| + iy v 6 ^(i + e )(i+e)y 
Manipulating on the first term, we obtain 

\n\ \r<\ / ,r. \ \ clog m 

p < d 1 + L_L \/c log 771 ' 



(l + e)^) 



Since \C\ <C m, we replace the first term by a constant less than e (indeed, 1 + o(l)) and fold it, along 
with c', into c" . We get 

. lr<l / , n , % \ clogm 

A n f(2 + e) (2+£ M 



p < c" f - J ^ 
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Since |C| = (2 + e)clog m, the previous equation simplifies to 

clogm 



P < c" y C log 



m 



(2 + e 



2(2+ £ )(i + £ )(i+ £ ) 



This last factor is 1 when e = 0, and decreases with increasing e. Thus, for any constant e > 0, the 
probability p is polynomially small when m is Q(y/n). The same result can be symmetrically obtained for 
intercalated elements; thus, we have at least clog m elements of each, with high probability. □ 

Note that since C is split evenly in expectation, the expected insertion cost is constant. 

Corollary 5 There are at least c log m support elements and at least c log m intercalated elements in each 
set of (2 + e)clogm contiguous elements with high probability. 

Proof. We are interested only in nonoverlapping sets of contiguous elements and there are m/(2 + 
e)clog m such sets. By Theorem 4 and the union bound, the claim holds. □ 



3.3 Summary 

We summarize our results as follows: The overall cost of rebalancing is 0(n) by Lemma 2. By Lemma 3, 
the cost of searching for the position of the ith element is O(logz), and then the overall searching cost is 
0(n log n). We proved in Lemma 1 that the insertion cost of the first 0(y/n) elements is 0(n) in the worst 
case. Finally, Theorem 4 shows that for sufficiently large c, no contiguous c log m elements in the support 
have (H-e)clog m intercalated elements inserted among them at the end of any round, with high probability. 
Thus, there is a gap within any segment of (2 + e)clogm elements with high probability. Therefore, the 
insertion cost per element for m = £l{yjn) is 0(log m) with high probability. The overall cost of Library 
Sort is 0{n log n) with high probability. 



4 Conclusions and related work 

We have shown that LIBRARY SORT outperforms traditional INSERTION SORT. There is a trade-off between 
the extra space used and the insertion time given by the relation between c and e. The lower the desired 
insertion cost, the bigger the required gap between elements at rebalance. 
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Library Sort is based on the priority queue presented in Itai, Konheim, and Rodeh [5]. Our analysis 
is a simplification of theirs. Moreover, we give high probability and expectation bounds for the insertion 
cost, whereas they only give expectation bounds. 

An algorithm similar to Library Sort was presented by Melville and Gries in [6]. This algorithm has 
a 1/3 space overhead as compared with the e space overhead of Library Sort. They point out that their 
running time analysis was too complicated to be included in the journal version. To quote the authors: "We 
hope others may develop more satisfactory proofs." . 

The idea of leaving gaps for insertions in a data structure is used by Itai, Konheim, and Rodeh [5]. This 
idea has found recent application in external memory and cache-oblivious algorithms in the packed memory 
structure of Bender, Demaine and Farach-Colton [1] and later used in [2-4]. 
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